Supplement for “Stochastic Convex Optimization: Faster Local Growth Implies Faster Global Convergence”

Theorem 1. Suppose Assumption 1 holds and F (w) obeys the LGC (6). Given δ ∈ (0, 1), let δ̃ = δ/K, K = dlog2( 0 )e, D1 ≥ c 0 1−θ and t be the smallest integer such that t ≥ max{9, 1728 log(1/δ̃)} D 1 0 . Then ASSG-c guarantees that, with a probability 1− δ, F (wK)− F∗ ≤ 2 . As a result, the iteration complexity of ASSG-c for achieving an 2 -optimal solution with a high probability 1− δ is O(cGdlog2( 0 )e log(1/δ)/ 2(1−θ)) provided D1 = O( c 0 (1−θ) ).